Earlier this month, scholar–educators and the HuMetricsHSS team came together at Michigan State University to better understand the values that shape how syllabi are crafted and evaluated.
As part of our Andrew W. Mellon Foundation–funded work to create use cases for applying values-based evaluation metrics across scholarly practices, this workshop explored the value of a syllabus from two angles: as a scholarly work in and of itself (e.g., What can the selection of texts and assignments tell us about the state of a discipline?), and as an indicator of the impact of other scholarly works (e.g., If a work is included in a syllabus, what does that mean for the author of the work? Is it worth adding to a tenure and promotion narrative to reflect one’s influence upon other instructors or the discipline as a whole?).
After introductions, we began our work by splitting into groups and analyzing randomly selected humanities syllabi to see if it was possible think about syllabi as textual features that could be visualized in an "analog" manner, by using paper, pen, and various craft supplies. This activity served as both an icebreaker and as a thought experiment for our group and the MSU Data Science Team, who will be working to develop a prototype app that can take a given syllabus and extract data from it to tell the author something about the values embedded in the syllabus.
Syllabi features identified included required texts lists (from which you could extract information like author nationality and gender), grading policies, class schedules, learning outcomes, and a host of other administrative content that usually contains implicit value statements (e.g., "Disability Accommodation" policies can signal an organizational commitment to Equity).
A major theme that emerged was that of sentiment analysis: What can we learn about an instructor or an institution's values from the punitive language, hedging statements, or optimism that is often embedded in the pages of syllabi?
In the afternoon, we introduced the HuMetricsHSS proposed core values to workshop participants, to further our thinking as a group about all the potential value-drivers for creating syllabi. Participants examined their own syllabi with these values in mind, reflecting upon the implicit and explicit values they integrated into their own teaching. One participant shared the realization that the language she chose for a performance class' syllabus assumed students' able-bodiedness, which in her opinion didn't align fully with the value of Accessibility. "I want my students to be able to say, 'I can do that! when they look at my class, and right now this syllabus wouldn't inspire all students to say that," she explained.
Day two of the workshop started with a frank discussion of what it means to us, as scholars, to have our work listed as readings in syllabi. The discussion quickly diverged into a roundtable on the various socio-technical challenges of discovering that one's work has been mentioned in the first place, including:
- A lack of structured data that would allow altmetrics services to properly scrape syllabi to find mentions of texts and digital scholarship
- A related dearth of syllabi made available openly and in a text-mineable format
- Lack of knowledge of standards for citing others in syllabi (and for citing others' syllabi in your own syllabi!)
- Full ownership over one's own syllabi as intellectual property (in many cases, the university or department claims ownership of a syllabus and gives it freely to other instructors to use to teach a course; in theory, sharing is good, but for adjunct professors who live with job precarity and are often seen as easily replaceable, ownership of one's own work is needed)
- The difficulty of recognizing the impact of well-crafted assignments, which are often not embedded in the syllabus but can significantly influence how a subject, topic, or text is taught throughout a discipline
- Well-meaning but uneducated department chairs and administrators, who don't always have the language to articulate why one's teaching contributions are important or why it is impactful to have been cited in others' syllabi
Context was also identified as necessary to be able to fully interpret one's inclusion in a syllabus. For example, there is a difference between one's work being listed as one of several readings for a class and in being the only reading for a class, around which an assignment is built.
Some solutions suggested to these and other challenges included:
- Sharing two versions of your syllabus: the one you present to students, and an annotated (perhaps draft) version that explains your thinking behind assigning certain texts and assignments
- University administrators teaching department chairs how to write a paragraph about one's teaching contributions in a letter of recommendation for tenure
- Contacting those you include in your syllabi to let them know that they are mentioned, and inviting conversation
We then ran an exercise to think through the values, practices, evidence and indicators, and institutional culture needed to fully embrace values-based syllabus design in academia. After journaling independently and then coming back together to discuss as a group, we talked through many ways in which values could be lived out in practice and thoughtfully "metricized" to help educators track one's progress and grow.
We ended the workshop with a discussion of the various places where "HuMetrics-like" conversations around teaching are already happening, which the HuMetricsHSS team will use to start conversations and grow our community.
Overall, our second workshop was a success — due in large part to the creativity and commitment of workshop participants. We are very thankful to all who attended and look forward to sharing the ultimate outcome of this workshop (the prototype app for finding values in syllabi) with the larger community in the months to come!
Post-workshop, we were interviewed as a group by project PI Chris Long on MSU's The Liberal Arts Endeavor podcast — check it out!